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DETAILED ACTION 

1 . Applicant's Amendment filed on 1/09/2006 has been entered with amended 
claims 1-3, 6, 8, 10-12, 15, 18-20, 23, 25, 27-29, 31-37, 40, 42, 44-46, 49, 52-59 and 
canceled claim 60. Claims 1-3, 5-12, 14-20, 22-29, 31-37, 39-46 and 48-59 are pending 
in this Office Action. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the claims 
under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was 
commonly owned at the time any inventions covered therein were made absent any evidence to the 
contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out the inventor and 
invention dates of each claim that was not commonly owned at the time a later invention was made in 
order for the examiner to consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), 
(f) or (g) prior art under 35 U.S.C. 103(a). 
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3. Claims 1-3, 5-9,18-20, 22-26, 35-37, 39-43,52,54, 56 and 58 are rejected under 
35 U.S.C. 103(a) as being unpatentable over Hoffman (US Patent 5,761,655) 
hereinafter Hoffman, in view of Schwartz et al. (US Patent 5,905,988) hereinafter 
Schwartz and in view of Shingo et al. (JP09-274659) hereinafter shingo. 

4. As per independent claims 1,18, 35, 58, Hoffman teaches a system to store, 
retrieve and display images of thumbnail size. The size of the thumbnails displayed can 
be changed to allow visually reviewing a large number or concentrating on a smaller 
number of but larger size images after reduction in the umber of thumbnails. 
Thumbnails as well as other image files and other related. text files are indexed and 
searched using keywords. The search can be performed, using personalized super- 
keywords, which are combinations of keywords and other file and data characteristics 
as keywords (col. 4, lines 8-21). Hoffman teaches the claimed "image storage means 
for storing a plurality of images" as the system reduces the original image size to the 
thumbnail image and stores the color data in the record (Fig. 2, col. 6, lines 18-21). 
Further, Hoffman teaches the claimed "selecting means for selecting a single image 
form the plurality of images, wherein the single image includes a plurality of objects" as 
the image files are retrieved, by conventional systems, thumbnails are created and 
displayed and it is well known that the single image includes plurality of objects (Fig. 1, 
col. 5, lines 64-66). Further, Hoffman teaches the claimed "input means (examiner 
interpreted input as clicking on keywords from Hoffman [see Fig. 15, col. 11, 
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lines 19-23]) for inputting management information (examiner interpreted management 
information as pixel data from Hoffman for each of the plurality of objects within single 
image [see Fig. 3, col. 6, lines 43]). Further, Hoffman does not explicitly teach 
describing an interrelationship with at least another object. However, Schwartz teaches 
"a word describing an interrelationship between one object with at least one other 
object" as one picture on the web along with any accompanying text describing the 
content picture (col. 13, lines 18-21). Thus, it would have been obvious to one of 
ordinary skill in the data processing art at the time of the invention, to combine the 
teachings of the cited references because Schwartz's teachings would have allowed 
Hoffman's method to provide adoptive serial presentation of the desired items (col. 3, 
lines 28-30). 

Further, Hoffman and Schwartz do not explicitly teach position and 
interrelationship between two objects. However, Shingo teaches the claimed "an 
interrelationship between one object with at least other object and position" as a 
predetermined criteria image in two-dimensional by the seek area in an input image, 
and it is each location of the seek area. The cross correlation function is added to the 
lengthwise direction and longitudinal direction of the seek area, and it is constituted as 
the image alignment approach characterized by making into the aggregate value of this 
longitudinal direction converted into the greatest value. The aggregate value is the 
square of the cross correlation function and is the image alignment value 
(Paragraph 0004). Thus, it would have been obvious to one of ordinary skill in the data 
processing art at the time of the invention, to combine the teachings of the cited 
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references because Shingo's teachings would have allowed Hoffman's method to 
improve cross-correlation coefficient method and it offers a greater performance in 
alignment of the exact image without increasing the operation time (paragraph 0003). 

Finally, Hoffman teaches the claimed "memory means for storing the 
management information input said input means in association with the single image" as 
the closest match is saved 172 as the pixel in the thumbnail image (at Fig. 5, col. 7, 
lines 2-14). 

5. As per dependent claims 2, 19, 36, Hoffman teaches the claimed "the 
management information further includes at least one of a general name of an object, a 
qualifier therefor, and a proper in thereof as the system allows the user to have a 
custom set of super keywords allowing a personalized search process without cluttering 
the search for other users (Fig. 19A-B, col. 3, lines 14-21 and col. 13, line 60 to col.14, 
line 20). 

6. As per dependent claims 3, 20, 37, Further, Hoffman teaches the claimed "the 
management information further includes information expressing a state (examiner 
interpreted state as color from Hoffman) [see Fig. 15, col. 1 1 , line 23] of an object in the 
single image" as a color selection 406 of screen 400 allows the colors of the image to 
be selected using color boxes (Fig. 15, col. 11, lines 23-25). 
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7. As per dependent claims 5, 22, 39, Hoffman teaches the claimed "a plurality of 
words can be specified as the qualifier" as keyword selection section 402 allows the 
user to indicate number of matches (Fig. 19A-B, col. 3, lines 14-21 and col. 13, line 60 
to col.14, line 20). 

8. As per dependent claims 6, 23, 40, Hoffman teaches the claimed "input means 
includes position designating (examiner interpreted designating as must match from 
Hoffman [see Fig. 15, col. 11, line 21]) means designating a position of an object in the 
single image, and display means for displaying an input window used to input 
management information concerning the object at the designated position" as one of the 
keyword selected is allowed to indicate as a must match keyword 415 by must match 
key depressed (Fig. 15, col. 11, lines 19-23). 

9. As per dependent claims 7, 24, 41 , Hoffman teaches the claimed "the position 
designating means designates positions of two mutually-related objects in the single 
image" as the must match keyword will indicate the closely related to the image in 
comparison to other selected key words (Fig. 15, col. 11, lines 19-23). 

10. As per dependent claims 8, 25, 42, Hoffman teaches the claimed "query input 
means for inputting query for retrieval, and image retrieving means for retrieving 
including an object image that meets the query inputted by said query input means" as 
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the disclosure is designed to allow fast matching of m out of n keywords (Fig. 15, 
col. 11, lines 39-48). 

11. As per dependent claims 9, 26, 43, Hoffman teaches the claimed "input means 
inputs supplementary information including at least one of imaging-related information 
of the single image, special object information thereof, category formation thereof, 
impression information thereof, time information thereof, place information thereof, 
weather information thereof, and event information thereof as three databases are 
involved in the keyword search and disclosed in detail (at Fig. 16, col. 11, lines 49-64). 

12. As per dependent claims 52, 54 and 56, Hoffman teaches the claimed "the 
management information is textual information" as the reminder of the space of the 
screen 210 to be used for text or other data, which needs to be combined with the 
images (Fig. 9, col. 9, lines 9-13). 

13. Claims 10-12, 14-17, 27-29, 31-34, 44-46, 48-51, 53, 55,57 and 59 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Hoffman (US Patent 5,761,655) 
hereinafter Hoffman and in view of Schwartz et al. (US Patent 5,905,9818) hereinafter 
Schwartz. 

14. As per independent claims 10, 27, 44, 59, Hoffman teaches a system to store, 
retrieve and display images of thumbnail size. The size of the thumbnails displayed can 
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be changed to allow visually reviewing a large number or concentrating on a smaller 
number of but larger size images after reduction in the umber of thumbnails. 
Thumbnails as well as other image files and other related text files are indexed and 
searched using keywords. The search can be performed using personalized super- 
keywords, which are combinations of keywords and other file and data characteristics 
as keywords (col. 4, lines 8-21). Hoffman teaches the claimed "image storage means 
for storing a plurality of images, wherein each image includes a plurality of objects" as 
the system reduces the original image size to the thumbnail image and stores the color 
data in the record and it is well known that the single image includes plurality of objects 
(Fig. 2, col. 6, lines 18-21). Further, Hoffman teaches the claimed "memory means for 
storing management information for each of a plurality of objects included in a single 
image in association with the single image" as (examiner interpreted management 
information for an image a as pixel data from Hoffman for the plurality of objects [see 
Fig. 3, col. 6, lines 43]) (Fig. 1, col. 5, lines 64-66). 

Further, Hoffman does not explicitly teach describing an interrelationship 
between two objects. However, Schwartz teaches the claimed "the management 
information includes a word describing an interrelationship between one object with at 
least one other object" as one picture on the web along with any accompanying text 
describing the contents of the picture (col. 13, lines 18-21). Thus, it would have been 
obvious to one of ordinary skill in the data processing art at the time of the invention, to 
combine the teachings of the cited references because Schwartz's teachings would 
have allowed Hoffman's method to provide adoptive serial presentation of the desired 
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items (col. 3, lines 28-30). Further, Hoffman teaches the claimed "query input means 
for inputting query for retrieval, wherein the query includes at least a word" as the 
invention allow fast match of m out of n keywords when hundreds of thousands of files 
are being managed and provides a unique method of searching keyword data (Fig. 15, 
col. 3, lines 14-21 and col. 11, lines 39-43). 

Finally, Hoffman teaches the claimed "retrieving means for retrieving an image 
including an object that meets the query inputted by said query input means, based on 
the management information stored in said memory means" as a dominant color and 
original height and width allow index search on three criteria (Fig. 7A col. 8, 
lines 56-59). 

1 5. As per dependent claims 1 1 , 28, 45, Hoffman teaches the claimed "the 
management information further includes at least one a general name of an object, a 
qualifier therefor, a proper noun thereof, and a position thereof as the system allows 
the user to indicate number of keywords matching and as well as designating certain 
keywords as must match keywords (Fig. 19A-B, col. 3, lines 14-21 and col. 13, line 60 
to col. 14, line 20). 

16. As per dependent claims 12, 29, 46, Hoffman teaches the claimed "the 
management information further includes information expressing a state (examiner 
interpreted state as color from Hoffman [see Fig. 15, col. 11, line 23]) of an object in the 
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single image" as a color selection 406 of screen 400 allows the colors of the image to 
be selected using color boxes (Fig. 15, col. 11, lines 23-25). 

17. As per dependent claims 14, 31, 48, Hoffman teaches the claimed "the qualifier 
is compromise of a plurality of words can be specified as the qualifier" as keyword 
selection section 402 allows the user to indicate number of matches (Fig. 19A-B, col. 3, 
lines 14-21 and col. 13, line 60 to col.14, line 20). 

18. As per dependent claims 15, 32, 49, Hoffman teaches the claimed "further 
comprising a position designating means for designating a position of an object of 
interest in the single image, and display means for displaying an input window used to 
input the management information concerning the object at the designated position" as 
one of the keyword selected is allowed to indicate as a must match keyword 415 by 
must match key depressed (Fig. 15, col. 11, lines 19-23). 

19. As per dependent claims 16, 33, 50, Hoffman teaches the claimed "wherein said 
* position designating means designates positions of two mutually-related objects in the 
single image" as the must match keyword will indicate the closely related to the image in 
comparison to other selected keywords (Fig. 15, col. 11, lines 19-23). 
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20. As per dependent claims 17, 34, 51 , Hoffman teaches the claimed "wherein said 
input means inputs supplementary information including at least one of imaging-related 
information of the single image, special object information thereof, category information 
thereof, impression information thereof, time information thereof, place information 
thereof, weather information thereof, and event information thereof as three databases 
are involved in the keyword search and disclosed in detail (at Fig. 16, col. 11, 

lines 49-64). 

21 . As per dependent claims 53, 55 and 57, Hoffman teaches the claimed "the 
management information is textual information" as the reminder of the space of the 
screen 210 to be used for text or other data, which needs to be combined with the 
images (Fig. 9, col. 9, lines 9-13). 

Response to Arguments 



22. Applicant's arguments filed on 1/09/2006 have been fully considered but they are 
moot in view of the new ground of rejection. Hoffman and Schwartz references are 
combined with the newly found Shingo teach each and every limitation as discussed 
above. 
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a) Applicant's argument stated as "Referring to Figure 9 of Hoffman, It is 
Applicants' understanding, however, that this text is nothing more than the file 
name of the image." 

In response to the Applicant's argument, Examiner respectfully disagrees 
because, one, the reference does not belong to claim 1. Two, regarding Hoffman 
Fig. 9 interpretation is not correct. He teaches as "Fig. 9 illustrates medium size 
thumbnail images where only a portion of the screen is being used for images, 
allowing the remainder of the screen to be used for text or other data, which 
needs to be combined with the images" (col. 9, lines 9-13). 

Conclusion 



23. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sathyanarayan Pannala whose telephone number is 
(571) 272-41 15. The examiner can normally be reached on 8:00 am - 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Charles Rones can be reached on (571) 272-4085. The fax phone number 
for the organization where this application or proceeding is assigned is (703) 872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
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For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

Sathyanarayan Pannala 

Examiner 

Art Unit 2164 



srp 

April 17, 2006 



